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PRODUCTIVITY ASSESSMENT: 
A STUDY OF FACULTY, ADMINISTRATOR, AND TRUSTEE PREFERENCES 

Introduction 

The definition, conceptualization, and measurement of educational pro- 
ductivity are dependent upon an undergirding view of the purposes of education 
and the manner in which it ought to be organized to accomplish these purposes. 
Quite simply, educational purpose and productivity measurement must fit to- 
gether like hand and glove. An inappropriate fit threatens to rend the glove 
or so cramp the encased hand as to render it clumsy if not ineffective. 

Three paradigms of educational purposes and resultant productivity 
schemes have been proposed elsewhere (Kirschling and Romney, 1977). These 
three paradigms are the: 

• Free-market learning enterprise 

• Planned-outcomes learning enterprise 

• Learning community 

Of these three paradigms, the former two currently dominate while the 
third deserves consideration not only to preserve an historically important 
element of higher education but also to provide an alternative form of learning 
opportunity. 

The planned-outcomes paradigm of postsecondary education is virtually 
guaranteed continued existence, at least for the foreseeable future and it is 
on this paradigm that the remarks of this paper are focused. At least the 
following factors underwrite the model's survival: 

• The enrollment bubble now emerging from the nation's colleges and 
universities and its associated ramifications for controlled retrench- 

' ment; 
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t The ubiquitous calls for accountability and demonstrations of 
productivity, one response to which involves providing evidence 
of goals (planned-outcomes) formulated, acted upon, and evaluated; 
and, 

• The continuing struggle for financial support in an area where 
advocates of other, equally beneficial social goals compete for funds. 

Given that the planned-outcomes paradigm of postsecondary education will 
continue to impact the conduct of a large sector of postsecondary education, 
a major concern is how to make the planning process and the resulting system 
more responsive to individual needs and expectations. Numerous authors have 
argued the necessity of participative decision-making, individual involvement 
in organizational goal setting and evaluation activities, and recognition 
of both individual and organizational needs to promote organizational health 
and effectiveness (Schein, 1969; Argyris, 1962; Schein and Bennis, 1965). To 
look briefly at the ramifications of a people oriented planning system was 
the leitmotif guiding the research study described in these pages. 

Research Design 

A series of crucial questions revolving around the theme of adding indivi 
perspectives to the planned-outcomes approach to higher education stems from 
concerns about the ability and interests of individuals to become so involved. 
Moreover, once involved, what do their views indicate about planned-outcomes 
(goals) and their evaluation. 

In order to examine these concerns, the author conducted a study entitled 
"Institutional Goal Achievement: Measures of Progress" with the following 
purposes: 
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(1) Assess the interests of three campus constituencies (faculty, 
administrators, and trustees) in being involved in institutional 
goal setting and assessment processes; 

(2) Examine the acceptability of specific goal areas (or planned- 
outcomes) to these three audiences in six kinds of institutions; 

(3) Assess perceptions as to which goal areas can be measured and how 
they can be measured, as viewed by trustees, administrators, and 
faculty; 

(4) Gather information regarding perceptions of appropriateness of 
specific kinds of information "as measures of progress toward the 
achievement of broadly-stated institutional goals; 

(5) Determine if the appropriateness of the measures of progress toward 
the achievement of institutional goal areas varies significantly 
among the three constituent groups; 

(6) Determine if the appropriate?iess of the measures of progress for 
different institutional gorJ areas varies significantly among six 
types of institutions. 

The study population consisted of 1150 faculty, trustees and adminis- 
trators from 45 colleges and universities throughout the country. These 
institutions were classified into six major categories, as follows: 

• Public Doctoral -Granting Institutions (7) 

• Private Doctoral-Granting Institutions (3) 

• Public Comprehensive Universities and Colleges (9) 
f Private Comprehensive Universities and Colleges (4) 

• Libral Arts Colleaes (12) 

• Two-Year Colleges and Institutes (10) 



Faculty and trustee respondents to the study were chosen randomly 
whereas administrators were selected based on their responsibility for the 
following activities or functions: 



( 1 
(•2 
{ 3 
( 4 
( 5 
( 6 
( 7 

-( 8 
{ 9 
(10 

(11 
(12 
(13 



Chief executive officer 
Chief academic officer 
Chief financial officer 
Chief student services officer 
Dean of the graduate school 

Dean of arts and. sciences (or general studies or transfer programs) 

Dean of professional schccfls (law, business, or occupational 
programs) 

Physical Sciences department chairperson 

Social Sciences department chairperson 

Humanities department chairperson 

Coimnuni cation Arts department chairperson 

Director of institutional research or analytical studies 

Chief planning officer 



The survey instrument to which these individuals were asked to respond 
consisted of two principal sections. The first contains demographic and 
identifying information about the respondent. The second dealt with measures 
of progress for twenty broadly-stated institutional goal areas. The goal 
areas were derived from the Institutional Goals Inventory (IGI) developed 
by and used with permission of the Educational Testing Service (ETS). The 
IGI consists of several specific goal statements in each of twenty institutional 
goal areas, thirteen of which are designated as outcome goal areas {academic 
development intellectual orientation ^ individual personal development ^ 
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.hmmism/altvuism^ cultimil/aesthetiQ ca^areness^ traditional religiousness^ 
vocational preparation^ advanced training^ research^ meeting local needs^ 
public service J social egalitarianism^' and social criticism/activism). The 
remaining seven goal areas are referred to as process goal areas by ETS 
{freedom^ democratic governance^ community^ intellectual/aesthetic environ^ 
menty innovative climate^* off-^campus learning^ and accountability /efficiency) . 

Associated with each IGI goal area on the study questionnaire were sets 
of four to eleven measures of progress or possible items of information to be 
used to demonstrate progress toward the achievement of the goal area in question. 
All in all, respondents, in addition to the twenty goal areas, were asked to re- 
spond tc 125 measures of progress. These measures were developed and contributed 
by the staffs of NCHEMS, WICHE, and the Higher Education Center of the School of 
Education at the University of Colorado. In addition, a panel of experts, iden- 
tified by the author and staff members at NCHEMS, reviewed and supplemented the 
list of measures. 

To complete the questionnaire, respondents were asked to rate the degree to 
which each goal area should be a goal of his or her institution. Secondly, the 
respondent was asked to indicate the degree to which each item of- information was 
considered to be an appropriate measure of progress for the goal area. The scale 
of appropriateness for each of these types of questions consisted of six categories 
inappropriate, low, below average, average, above average, and high. In the 
scoring process, these were assigned point values of one through six, respectively. 
An example of measures of progress and the goal areas to which they relate is 
illustrated in Figure 1. 



* Note that ETS gave this goal area the name of innovation. Pilot test 
activities indicated that Innovative Climate was more descriptive. 



Intellectual Orientation. (Illustrative goals in this area include attitudes 
about learning and intellectual work; familiarity with research and prob* 
lem-solving methods; the ability to synthesize knowledge from many 
sources, and/or the capacity for self-directed learning and a commitment 
to lifelong learning.) 


should 
be a 
goal 


□ 


□ 


□ 


□ 


□ 


□ 
















1. Student and/or former student perceptions and evaluations about 
their attitudes and beliefs toward new and different intellectual work . 


□ 


□ 


□ 


□ 


□ 


n 


^. oiUQeni anu/or lormer siuucni percepiinns anu evaiuaiions oi ineir 
interest in continued self-initiated study and inquiry (follow-up ques- 
tionnaire). 


□ 


□ 


□ 


□ 


□ 


□ 


3. Student and/or former student ability to create origrnal perspectives, 
explanations, and implementations (standardized instruments). 


□ 


□ 


□ 


□ 




□ 


4. Student and/or former student ability to formulate and analyze prob- 
lems (standardized instruments). 


□ 


□ 


□ 


□ 


□ 


□ 


5. Quality of graduate-level theses or dissertations completed (institu- 
tional records]. 


□ 


□ 


□ 


□ 


□ 


□ 


6. Continuing, active intellectual involvement of former students in 
other than formal, advanced study (follow-up study). 


□ 


□ 


□ 


□ 


□ 


□ 


7. Student evaluations of courses (course evaluation studies). 


□ 


□ 


□ 


□ 


□ 


□ 


8. Scholarly works produced by students and/or former students that 
are considered suitable for publication (institutional study). 


□ 


□ 


□ 


□ 


□ 


□ 


9. 


□ 


□ 


□ 


□ 


□ 


□ 



FIGURE 1 

MEASURES OF PROGRESS RELATED TO GOAL AREA B: 
INTELLECTUAL ORIENTATION 



Study Findings 

The study produced a wealth of infonnation, only a small portion of 
which has been thoroughly examined. A detailed examination of the results 
and conclusions obtained thus far are recprded elsewhere (Romney, 1976). Many 
of the results, which proved to be unexpected, interesting, and worthy of high- 
lighting in this type of report, are summarized below: 

• Conducted during the summer months of the year (1976) when many insti- 
tutions and faculty operate under reduced schedules, workloads, and 
expectations, the survey achieved a surprisingly high response rate. 
As illustrated in the following table, the rate of return exceeded 
74 percent in the aggregate for all three types of respondents and 
six types of institutions. g 
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TABLE I 



■ RESPONSE RATES FOR THREE TV'PES OF RESPONDENTS 
IN SIX TYPES OF INSTITUTIONS 



CELL KEY: 



MO. RESPONDED 
PERCENT RESPONDED 



TYPE OF INSTITUTION 



TYPE OF 


PUBLIC 


PRIVATE 


PUBLIC 


PRIVATE 


LIBERAL 


Tl'iO-YEAR 




RESPONDENT 


DOCTORAL 


DOCTORAL 


COMPREHENSIVE 


COMPREHEilSIVF 


ARTS 


COLLEGES 


TOTAL 




12 


3 


7 


7 


16 


19 


64 


TRUSTEE 


















602 


33% 


26% 


64% 


44% 


63% 


48% 




71 ; 


28 


73 


33 


63 


75 


343 


ADMINISTRATOR 


















90% 


82% 


72% 


79% 


82% 


89% 


82% 




77 


62 


62 


74 


90 


76 


441 


FACULTY 


















77% 


62% 


62% 


74% 


90% 


76% 


74% 




160 


93 


142 


114 


169 


170 


848 


TOTAL 


















80% 


65% 


62% 


75% 


79% 


79% 


74-::; 



0 The low response rates of trustees, in addition to the comments made 
on many of the returned questionnaires, suggest an apparent lack of 
familiarity on the part of trustees with the institutions and their 
operations. Many were able to address the issue of goal area ap- 
propriateness but regarded the determination of appropriate measures 
of progress beyond their knowledge, abilities, or responsibilities. 
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If the specification of measures of progress for institutional goal 
areas is one of the important stepping stones to translating these 
goals into statements of measureable objectives, then one must wonder 
what the role of trustees in such a process would be. Moreover, 
trustees, perhaps more than any other institutional constituency, 
have an interest in evidence of progress toward achieving institutional 
goals. Thus, that trustees believed themselves to be inappropriate 
respondents and yet are some of its most obvious beneficiaries bespeaks 
the need for an educational effort on the part of institutions to 
instruct and involve their boards in institutional goal selection and 
evaluation. 

With regard to the goal areas, participants from Public and Private 
Doctoral -Granting institutions place high emphasis on the roles 
traditionally held for these institutions; that is, research and the 
pursuit of the full academic/intellectual life. Trustees mostly agree 
except where obligations of a higher institution to its benefactors 
are concerned. Ratings of these types of goal areas seems to be more 
pronounced in the trustee ratings than they are in the ratings of the 
other groups. For example, the goal areas of Vocational Preparation 
and Accountability /Effieiency both appear in the trustee ratings but 
not In the seven most highly preferred goal areas of faculty or admin- 
istrators. Moreover, the ratings of appropriateness of goal areas by 
faculty in Public and Private Doctoral -Granting Institutions underscore 
their stress on the full academic life and desire that it be protected 
by the concepts and policies of academic freedom. In summary, there is 
a strikingly similar emphasis given to the seven goal areas considered 
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to be most appropriate by all three types of respondents in the Doctoral - 
Granting Universities. 

• Faculty preferences in Public Comprehensive colleges vary only slightly 
from those of faculty in Doctoral -Granting Institutions, implying at 
least a similarity of 'expectations and ideals if not a latent desire 

for what is perceived to be the full academic life more readily attainable 
in Doctoral -Granting institutions. 

• Emphasis on "the individual" is implicitly expressed in the seven most 
appropriate goal area^ of respondents from Liberal Arts and Two-Year 
Colleges. Whereas respondents from Liberal Arts Colleges emphasize 
the importance of the individual from an intellectual /academic per- 
spective, Two-Year College and Institutes respondents voice their con- 
cern for the individual in terms of Vocational Preparation and Social 
Egalitarianism goals. 

• Generally, the appropriateness of measures of progress for demonstrating 
institutional goal achievement did not vary across the faculty, trustees, 
and administrators. There were only seven instances in which views of 
the three types of respondents differed significantly. Trustees and 
administrators differed only with regard to two measures: 

(1) Scholarly works produced by students and/or former students that 
are considered suitable for publication (trustees lower than 
administrators); 

(2) Institutional policies and procedures developed to protect the 
exercise of academic freedom by faculty students (trustees 
lower than administrators). 
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Trustees and faculty ratings of appropriateness differed only for one 
measure. Institutional policies and prooedures developed to protect the 
exoerise of aoadem6if'reedom by faculty and students (faculty significantly 
higher than trustees). 

Administrator and faculty ratings of the appropriateness of measures of 
progress differed significantly only with regard to five measures: 

(1) Satisfaction of currently enrolled students or recent graduates 
with their academic development (administrators rated this as 
being more appropriate than did faculty), 

(2) Student/ faculty ratios (faculty rated this measure higher than did 
administrators) , 

(3) Amount of release time granted to faculty members for community 
service (faculty higher than administrators), 

(4) Availability and use of an institutional information system 
(administrator ratings higher than those of faculty members), 

(5) Satisfaction of students and former students with the extent and 
nature of their educational ea:^erienoe and subsequent employment 
(administrators higher than faculty). 

t The results suggest that if the following fourteen types of information 
were collected, progress could be demonstrated toward the achievement 
of the seven most important goal areas for each of the six institutional 
types. Moreover, this evidence would be acceptable generally to at 
least the faculty, administrators, and trustees in each type of institution. 
( 1) Student ability to apply knowledge 

( 2) Continuing active intellectual involvement of former students 
other than formal:, advanced study 
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( 3) Course offeiri^nga and institutional opportunities pertaining to the 
development of individual goals ^ values ^ and personal growth 

(4) Students and/ or former students expressing concern for human 
welfare and well being 

( 5) Employer satisfaction with former student vocational or professional 
training 

^ 6) Scholarly works produced by graduate students and/or former graduate 

students considered suitable for publication 
( 7) Basic research publications or other results of scholarly effort 

produced by students or faculty members during the past year 
( 8) Evaluations and perceptions of members of the cormrunity regarding , 

the quality of institutional services available to them 
( 9) Existence of special courses and programs to meet the needs of 

particular groups of students 
(10) Institutional policies and procedures developed to protect the 

exercise of academic freedom by faculty and students 
(n) Attendance and participation by faculty in the faculty senate or 

similar body 

(12) Faculty and staff perceptions and evaluations of internal morale 

(13) Student and/or faculty attendance at cultural activities sponsored 
by the institution 

(14) Impacts of modifications made in courses and programs 

t An examination of the fourteen measures listed above reveals a decided 

... ^ 

lack of compatibility with measures currently used to provide evidence 
of productivity. For example, in a study conducted to determine which 
"outcome measures'* of community colleges are collected by state agencies, 
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Kinnison (1975) found that most are required to supply information on the 
number of students enrolled, full-time equivalent students, and degrees 
granted. To such a list one could add "grade-point averages" and "student/ 
faculty rat-^os" as typically collected measures. Yet none of these 
measures was rated as highly appropriate in this study. In fact, the 
measures now in use tend to conform more in orientation and substance to 
those rejected by the participants. This apparent conflict can have some 
important implications for institutional management and productivity, as 
well as for statewide control and coordination. If the theme of enumer- 
ation survives as the way to provide evidence of accountability, insti- 
tutional administrators and faculty may begin to operate in accordance 
with radically different incentive structures. Emphasis on quality and 
impact would pale in the light of degree production and "body counts." 
Indeed, it is suspected that most incentives would operate to maximize 
degree production and body counts. In such a situation, all participants 
tend to lose in terms of impact and satisfaction. The measures found to 
be appropriate pieces of evidence provide an empirically substantiated 
base for such a change. 

t Respondents generally reacted negatively to those measures that implied 
any authenticity for students evaluating courses or providing comments 
on the relevancy of course materials. Such a rejection of student input 
suggests that many respondents wish to remind us who they are and hold 
to the belief that course development and curriculum design should reside 
within the purview of the organization, where the authority, responsibility, 
and expertise are currently found. That student evaluations and comments 
are generally treated with scepticism suggest also that the results of 
student-driven course and faculty evaluation mechanisms may be largely 

unheeded. 1 A 




One of the more important and obvious conclusions stems from an 
analysis of the types of information which were rated as being 
appropriate for measuring progress toward institutional goal achieve- 
ment. Analysis of the study findings leads to the conclusion that 
measures which pertain to the quality of the academic experience, 
the effectiveness of performance, and the utility of and satisfaction 
with skills acquired are deemed more appropriate by virtually all 
categories of respondents for measuring progress than are those which 
are process oriented or are traditional, quantitative measures. Although 
there were exceptions, the emphasis of participants was on preferences 
for measures of impact and satisfaction. 

A number of quality and impact measures, along with many others, were 
also rejected by the study participants. Although the rationale for 
giving any measure a particular rating was not studied, logic suggests 
at least five reasons underlie giving any measure a low assessment of 
appropriateness: 

(1) Face Validity I Apparent inapplicability of V:he suggested measure 
to the goal area 

(2) Lack of Background: Apparently inappropriate source for the 
information 

(3) Data Problem: Apparent infeasibility of collecting the proposed 
information 

(4) Institutional Diversity: Conflicts of institutional style with 
the style implied by the measure 

(5) Obtuseness: Lack of definitional clarity 
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To overcome some of these reasons for rejecting measures requires little 
more than concern for and skill in preparing the definitions and justify- 
ing that the measure indeed relates to the goal area in question. At the 
heart of overcoming the other reasons for rejecting measures are found 
the ghosts .of territorial imperatives and personal preference functions, 
topics which have occupied the minds of scholars for years. 

In a different vein, the results of the study suggest some conclusions 
regarding the development of goals and measures of progress for these 
goals. Respondents from six types of institutions from all parts of the 
country were participants in this study. In general, it can be said that 
concensus was reached across all types of institutions as to the ap- 
propriateness of some goal areas and measures of progress within these 
goal areas. In some cases, consensus regarding goals and their measures 
was restricted to agre^^^^ent within particular institutional types. There- 
fore, it is suggested that the process of selecting institutional goals 
and appropriate measures of progress can utilize consensus-building 
techniques and that these techniques are useful within individual insti- 
tutions, within systems of similar types of institutions , and across a 
conglomeration of several types of institutions. The number of goals 
and appropriate measures thought to be appropriate most likely will 
decrease as the diversity of institutions involved increases. Yet the 
task of identifying perceptions of common purposes and of how to measure 
progress toward their achievement seems to be feasible as well as necessary. 
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Conclusions 

With regard to those whose responsibilities lie in the college personnel 
field, the results of the study as well as the guiding paradigm of the organ- 
ization have important implications for the conduct of their efforts. 

Even when operating under the influence of the planned-outcomes paradigm, 
it must be recognized that many groups of individuals have both the interest 
and capability to provide direction to an operation and determine how its 
results should be assessed. Most importantly, persons responsible for personnel 
activities and programs ought to; 

• Be aggressive in insisting that their perspectives are recognized and 
accounted for in those forums where directions are set and assessment 

^schemes are determined; 

t Consider their role in facilitating the setting, achieving, and 
assessing of goals end outcomes of those constituencies whom they 
serve; 

t Consider the impl ication:^ of advocating the interests of any one 
constituency over those of others; 

0 Act as linkages between constituencies served and others in the 
system in order to ensure broader input into the goal setting and 
evaluation process; 

• Remember that in addition to being an organizational measure, pro- 
ductivity has equal if not greater meaning in individual terms and 
that individual productivity, as demonstrated in this study, is 
most preferably expressed in terms of satisfaction, impact, and 
quality of the learning experience, regardless of organizational 
goals and their achievement; 
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• Consider organizing their data collection requirements in terras 
similar to those expressed by the respondents in this study (i^e. 
replace numerical counts with indicators of impact, utility, and 
satisfaction). 
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